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Abstract 

We propose a model for description of activity-dependent evolution and self-wiring between bi- 
nary neurons. Specifically, this model can be used for investigation of growth of neuronal connec- 
tivity in the developing neocortex. By using computational simulations with appropriate training 
pattern sequences, we show that long-term memory can be encoded in neuronal connectivity and 
that the external stimulations form part of the functioning neocortical circuit. It is proposed 
that such binary neuron representations of point-like cortical neurons fail to reproduce cytoar- 
chitectural differences of the neocortical organization, which has implications for inadequacies of 
compartmental models. 
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I. INTRODUCTION 



The nature of processes, underlying the brain's self-organization into functioning neo- 
cortical circuits remains unclear. For theoretical description of activity-independent and 
activity-dependent emergence of complex spatio-temporal patterns, and establishment of 
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neuronal connections, various methods have been proposed 
oretical investigations give a fresh understanding of the observed phenomena and stimulate 
discovery of emergent features of neural circuits. In this paper, we propose a new theoretical 
approach for description of activity-dependent evolution and self- wiring between binary neu- 
rons. We propose a model based on the following experimental data (l)-(3) and hypotheses 
(4)-(5): 



1. Development of neuronal connectivity dependents on the neurons activity 
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2. Direction of motion of the growth cones is controlled by diffusible chemicals -axon 



guidance molecules (AGM) 
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3. Axon's growth rate dependence on the neuron's activity 



4. Depolarization causes neurons to release axon guidance 



[3, 13, 15, 3 



5. Type of neuronal connectivity is determined postsynaptically during synaptogenesis 



□ 



A state of all cells at some moment we shall call the activity pattern. Sequential alter- 
nation of activity patterns we shall call activity dynamics. We suppose that each neuron 
receives an external signal from a sensory cell (optical, taste, smell, ets.). Patterns of exter- 
nal signals we will call training patterns. We demonstrate an example of training process 
by stimulation of the developing net by the sequence of training patterns. The sequence 
of training patterns can be considered as a training program. The goal of the training is 
creation of complex connections structure between initially disconnected neurons. For math- 
ematical description of net's electrical activity we use the simplest model Q]. We take into 
account only properties which are most important for description of the activity-dependent 
self-wiring. 
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II. MATHEMATICAL FRAMEWORK 



In this section we consider a mathematical basis for description of the activity - dependent 
self-wiring in neural nets. For mathematical description of neurons states and activity 
patterns, we use simple representation from modern neurobiology and artificial neural nets 
(ANN) theory. At state of rest neuronal membrane is polarized. When the membrane is 
locally depolarized up to the certain value of transmembrane potential by opening the ligand 
activated synaptic ion channels, then the voltage gated ion channels are opened which causes 
the generation of the action potential. The action potential travels along the axon and causes 
local depolarization of another neurons through synapses. We suppose that each cell at each 
moment of time can be in two states: active, Si(tj) = 1, or inactive, Si(tj) = 0. For real 
neurons the active state can be regarded as the real neuron's single spike or a sequence of 
spikes with frequency above some value. 

We consider the z-th neuron's cell body as a spherical object with center at the point with 
the radius- vector in extracellular environment. For implementation activity-dependent 
release of AGM in our model, we suppose that neurons release AGM at a moment of depo- 
larization. For simplification we assume that all neurons fire and release AGM synchronous, 
depending on their state. AGM which have released by neurons diffuse through the extra- 
cellular space, then bind axon's growth cones receptors and control their growth. When we 
consider AGM release by neurons, we neglect geometrical properties of them and consider 
them point sources of AGM. 

According to our model if the cell is in active state, Si(tk) = 1, it releases some amount 
of AGM. We suppose that all neurons release the unit amount of the one type AGM which 
causes only attraction of growth cones. 

For description of AGM diffusion process we use a simple diffusion equation. The concen- 
tration of AGM, Cij, released by the i-th cell at the moment tj can be found as the solution 
of the equation 

^ = D 2 Adj - k ClJ (2.1) 

with the initial conditions Cy(r, rj, t/) = S(r — Ti)Si(tj). Here D and k are AGM diffusion 
and degradation coefficients in the intracellular medium. We consider here the case without 
boundary conditions. The solution of this equation, describing the concentration of AGM 
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at the point r at the time t is 



Si{tj) 



r — r< 
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Total concentration of AGM at the point r can be found by summation of concentrations of 
AGM which were released by each cell 

N k 

C(r,t) = EE^(r,r J ,t,t i ). (2.3) 
i=i j=i 

According to the experimental data we suppose that a growth cone will move only if its 
soma is at inactive state, and the force acting on it is proportional to AGM concentration 
gradient VC at the growth cone's position. Therefore, the equation of motion of the n-th 
neuron's k-th growth cone, described by the radius-vector g£ in the chemical field can be 
written in the following form 



XVC(glt)[S n (t)-l}. (2.4) 



dt 

Here A is a coefficient describing axon's sensitivity and motility. Taking into account the 
expression for total concentration from Eqn. (J2.3)) and using it into Eqn. (|2.5|) we obtain 

J n N k 

-f- = X[S n (t) - V Cw (gJ,r i ,M i ). (2.5) 

i=l j=l 

From Eq. (|2.2|) we find the expression for gradient of AGM concentration cy in the form 
below 



r - i\ 
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Vc,j(r ' r " M,) = -i6DM( / -i j )} exp {- kit - ti] - TdhTT-)) ■ < 2 - 6 » 

If a some growth cone is close to the another cell's soma, i.e if |g£ — < e (e can be 
considered as the soma's geometrical radius) then synaptogenesis process takes place, and 
synaptic connection between these neurons will be established. Type of the neuronal con- 
nections between z-th and n-th neurons we describe by using synaptic weights w ni i (w n> i = 1 
means excitatory and w n i = —1 inhibitory connections). In our model the type of synaptic 
connection established between neurons depends on the state of postsynaptic cell at synapto- 
genesis moment (if Si(tk) = 1 then w n ^ = 1, else w n> i = — 1). For implementation network's 
electrical activity dynamics several models can be used (firing rate models, integrate and 
fire neurons, etc.). For simplicity we will exploit the simplest one. We assume that the state 
of each cell at next moment of time is determined by states of another neuron, and neuronal 
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FIG. 1: A flowchart of self- wiring process 

connections weights, and external signal. Each postsynaptic cell integrates inputs coming 
from all presynaptic neuron and an external signal Sf xt 

N 

S=Y,Wn > iS n (ti)+Sr t , (2.7) 
n=l 

and the state of each neuron at next moment is determined by following rule: if s > then 
Si(t i+1 ) = 1, else Si(t i+1 ) = 0. 

This model gives a closed set of equations describing AGM's release and diffusion, and 
axons grow and neuronal connections establishment as well as the net's electrical activity 
dynamics. In the flowchart (Fig. 1) the back loop in the neural net's activity-dependent 
development and self-wiring are shown. The concentration of AGM in the extracellular 
space is controlled by neurons state. Growth and movement of growth cones is managed 
by the concentration gradients of AGM. Growth cones can make neuronal connections with 
other neurons and change the network's connections structure which change the network's 
activity. 



5 



III. NUMERICAL ANALYSIS 



In this section we present the result of the numerical simulation of the self-wiring net 
presented above. For testing possibilities and properties of self-wiring nets we use the net 
with 27 neurons, N = 27, placed at the points with radius-vectors r x = (0, 0, 1), r 2 = (0, 0, 0), 
r 3 = (0, —1, 0), ... = (1,1, —1) (see Fig. 3). Each neuron has 27 growth cones, which can 
be considered as branches of its axon. Initially all growth cones are located near the soma 
g£ = rfc + e, |e| < 0.01, and all synaptic weights equal to zero [wik — i,j — 1...N). 

We can write the Eqn. ()2.6|) for AGM's gradient using a discrete time t = nAt (present 

time), t m = mAt (release time) in the following form 

^ / * SAnAt)(r - n) ( , |r-rJ 2 \ , , 

Vc r, r i( mAt, nAt) = % \ — ^ exp —kt - — ^ 1 — I — , 3.8 

V ; 16£> 5 7rf ((n-m) At) § \ 4D*{n-m)At)' V ; 

where At is a time interval between the net's iterations. For simplification we set At = 1 

and rewrite the Eqn. ()3.8|) in terms of indices m, n 

^ , \ Si(n)(r — ri) ( , |r — rJ 2 \ 

Vc m ,„ r, r, = * {) ~r" exp —kt - ' r . 3.9 

For numerical integration a growth cone equation of motion (|2.5|) we have to rewrite it in 
terms of the discrete time. Using the time interval At = 1 we obtain the recursion relation 
which allow us calculate the spatial position of each growth cone at the moment k + 1 in 
terms of the k-th moment 

N k 

g fc+1 = g fc + X[S n (t k ) - 1]E E Vc m , fc (g k , ri ). (3.10) 

i=l m=l 

In above equation we have omitted indices, describing a cell's and the growth cone's 
number. The value At = 1 is not critical for numerical calculations of the model and 
it was used only for simplification point of view. Different values of parameters D, k, A 
gives different connectivity patterns between neurons, because these parameters characterize 
growth cone's movement speed, and AGM's acting distance, and etc. Basic properties of 
the net have remained unchanged. For all subsequent numerical calculations we used the 
following values: D = 0.1, k = 0.1, A = 10. 

In Fig. 2 we present the contour plot of the section by plane z = 0.02 the static AGM 
concentration when the 6-th, 7-th, 15-th, 16-th, 19-th neurons are in active state. The 
arrows reproduce the concentration gradients which means the axons movement directions 
for the particular place. 
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FIG. 2: The contour plot of target-derived AGM concentration. The concentration of AGM is 
higher in lighter regions of the figure. The arrows reproduce the concentration gradients which 
means the axons movement directions for the particular place. 

By using the appropriate training patterns sequence the net is built with inhibitory 
and excitatory connections which is evolved in oscillatory manner without any external 
stimulation. As an example we present the results of the network training process which 
was obtained by using the following training pattern sequence: Sff* = 1 for 10 < n < 192; 
Slf = 1 f o r 192 < n < 360; Sf* = 1 for 360 < n < 515; S^ 1 = 1 for 525 < n < 576; 
Sl xt = 1 for 580 < n < 875; Sf = 1 for 580 < n < 880; S^ 1 = 1 for 580 < n < 880; 
S^f* = 1 for 580 < n < 880. In Fig. 3 we reproduce three dimensional picture of the 
state of the net at the simulation beginning. Individual neurons (whilst without neuronal 
connections) depicted as spheres and its states depicted by color (bright - active state, dark- 
inactive). The travelling axon's branches are depicted as thin curves. One can see from this 
figure, how axons grow toward the active cell. When the growth cone reaches the soma of 
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FIG. 3: Activity-dependent evolution at the beginning of simulation (n = 110), see Section III. One 
of the cell has stimulated by the external signal S\ f 1 = 1 (bright sphere) , and other neurons are at 
inactive state. The axon's branches of the nearest neurons have started to grow in the direction of 
the active cell because it releases AGM. 



another neuron, two neurons became connected and axon's connected branch is depicted by 
thick curve (Fig. 4). The synapse type (inhibitory or excitatory) is depicted by color of a 
curve (bright - excitatory, dark- inhibitory). When the branch of the same axon reaches the 
another cell, which has already connected with it, this branch will be deleted. The neural 
net obtained by training is shown in Fig. 5. 

Further training of this net is a hard problem because the internal oscillation will influence 
to the training process. The oscillating patterns of neuronal activity will cause establish- 
ment of new connections, even after switching off the external stimulation, and can cause 
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FIG. 4: Activity-dependent evolution at n = 680. Between some of the neurons inhibitory (i) and 
excitatory (e) connections have been established. 

uncontrolled self-wiring. In forthcoming papers we plan to develop programming (training) 
methods in order to build the net with initially defined connections. 

We can gain access to the patterns stored in the net by stimulating them by external 
patterns. Because of an external stimulation will cause activation of neurons and release 
of AGM we conclude that access to the memory can change the connectivity pattern, i.e. 
patterns stored in the memory. 
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IV. CONCLUSIONS 



In this paper we presented a mathematical model describing processes of self-wiring in 
neural nets. The model can give us a new understanding of real neural nets development 
and functioning. By using the computer simulations we showed that the long-time memory 
can be encoded in neuronal connectivity and in what way the external stimulations builds 
a functioning neural network. 

Hypotheses proposed here should be tested experimentally (in vivo, in vitro), for ver- 
ification the possibilities of presented model application for qualitative and quantitative 
description of a neuronal integration processes in a real nets. Our approach gives only gen- 
eral modeling scheme of activity-dependent network formation processes and to accordance 
with experimental data the model can be complicated and a particular rules describing 
separate processes can be changed. 

The theoretical framework developed here, can be used for description the development 
of a particular set of neurons constituting the neural system, which can be located in the 
media containing also another type neurons and glia cells. In our model axon's branches 
began to growth directly from the soma. In a real nets growth cones can be guided at the 
growth beginning by another mechanisms (genetically determine contact adhesion, etc) and 
then by the mechanism presented in this paper. Geometrical properties and activity of real 
neurons are more complicate and they have dendrites. Here we used the most simple repre- 
sentations of hypothetical neurons (large spherical soma without dendrites ) and its activity 
(binary neurons). Using the approach described here it is possible to construct a model for 
description real particular networks, using more realistic models of neurons (ingrate-and-fire, 
Hodgkin-Huxley, etc) and taking into account neurons geometrical properties and dendrites. 
We suppose that the model presented can be used for description self-wiring processes in a 
developing (embryonal) as well as adult neuronal systems. 

Cytoarchitectural differences between different cortical areas are the result of differences 
in distributions of neuronal phenotypes and morphologies. Self-wiring between binary neu- 
rons fails to reproduce cytoarchitectural differences of the neocortical organization, which 
has implications for inadequacies of compartmental models. In order to reproduce cytoar- 



chitectural differences, axonal and dendritic morphologies of single neurons 
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integrated into the model. 
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FIG. 5: Activity-dependent evolution at (n = 910). By using the training program the complicated 
connections pattern between neurons have been created. The network has following neuronal 
connections: 1010,11 = 1, 1012,11 = 1 ,102,11 = 1, u>i3,i4 = -1, 1015,14 = -1, 105,14 = ~ l - ^23,14 = -1, 

^17,14 = -1, 1011,14 = 1, 106,5 = -1, 104,5 = "I, ^8,5 = -1, 102,5 = 1, «>ll,5 = ~1) ^17,5 = ~1, 

1015,5 = "I, 1013,5 = "I, 1012,5 = 1, ™14,5 = 1, ^10,5 = 1, «>14,11 = ~1, 1020,11 = ~1, ™1,2 = ~1, 

^3,2 = -1, W7,8 = -1, 1^23,24 = ~1, 1022,23 = "1, 1^26,23 = ~1, 109,6 = ~1, U>15,6 = ~1, ^9,8 = ~1, 

1017,8 = -1, 103,6 = -1, 1012,2 = ~1, 10l4,2 = ~1, 105,11 = ~1, 105,6 = ~1, ^14,23 = ~1, ^17,6 = "1, 

^12,6 = -1, 1^14,6 = -1, ^10,6 = -1, 1020,6 = ~1, W>20,2 = ~1, 1013,16 = ~1, 1^20,23 = ~1, 
1^10,2 = -1, 1013,11 = -1, W5,2 = 1- 
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